Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons
نویسندگان
چکیده
This study evaluated measures for making comparisons of errors across time series. We analyzed 90 annual and 101 quarterly economic time series. We judged error measures on reliability, construct validity, sensitivity to small changes, protection against outliers, and their relationship to decision making. The results lead us to recommend the Geometric Mean of the Relative Absolute Error (GMRAE) when the task involves calibrating a model for a set of time series. The GMRAE compares the absolute error of a given method to that from the random walk forecast. For selecting the most accurate methods, we recommend the Median RAE (MdRAE)when few series are available and the Median Absolute Percentage Error (MdAPE) otherwise. The Root Mean Square Error (RMSE) is not reliable, and is therefore inappropriate for comparing accuracy across series.
منابع مشابه
A Wavelet Support Vector Machine Combination Model for Daily Suspended Sediment Forecasting
Abstract In this study, wavelet support vector machine (WSWM) model is proposed for daily suspended sediment (SS) prediction. The WSVM model is achieved by combination of two methods; discrete wavelet analysis and support vector machine (SVM). The developed model was compared with single SVM. Daily discharge (Q) and SS data from Yadkin River at Yadkin College, NC station in the USA were used. I...
متن کاملOn the Selection of Error Measures for Comparisons Among Forecasting Methods
Clements and Hendry (1993) proposed the Generalized Forecast Error Second Moment(GFESM) as an improvement to the Mean Square Error in comparing forecasting performance across data series. They based their conclusion on the fact that rankings based on GFESM remain unaltered if the series are linearly transformed. In this paper, we argue that this evaluation ignores other important criteria. Also...
متن کاملCorrespondence On the Selection of Error Measures for Comparisons Among Forecasting Methods
Clements and Hendry (1993) proposed the Generalized Forecast Error Second Moment (GFESM) as an improvement to the Mean Square Error in comparing forecasting performance across data series. They based their conclusion on the fact that rankings based on GFESM remain unaltered if the series are linearly transformed. In this paper, we argue that this evaluation ignores other important criteria. Als...
متن کاملShort Term Load Forecasting Using Empirical Mode Decomposition, Wavelet Transform and Support Vector Regression
The Short-term forecasting of electric load plays an important role in designing and operation of power systems. Due to the nature of the short-term electric load time series (nonlinear, non-constant, and non-seasonal), accurate prediction of the load is very challenging. In this article, a method for short-term daily and hourly load forecasting is proposed. In this method, in the first step, t...
متن کاملA new accuracy measure based on bounded relative error for time series forecasting
Many accuracy measures have been proposed in the past for time series forecasting comparisons. However, many of these measures suffer from one or more issues such as poor resistance to outliers and scale dependence. In this paper, while summarising commonly used accuracy measures, a special review is made on the symmetric mean absolute percentage error. Moreover, a new accuracy measure called t...
متن کامل